ON AN ARGUMENT OF J. F. CARDOSO DEALING WITH 
PERTURBATIONS OF JOINT DIAGONALIZERS 
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Abstract. B. Afsari has recently proposed a new approach to the matrix joint 
diagonaUzation, introduced by J.-F. Cardoso in 1994, in order to investigate 
the independent component analysis and the blind signal processing in a wider 
prospective. Delicate notions of linear algebra and differential geometry are 
involved in the works of B. Afsari and the present paper continues such a line of 
research, focusing on a theoretical condition which has significant consequences 
in the numerical applications. 



1. EJD PROBLEM AND RELATIVE GENERALIZATIONS 

Several branches of the digital signal processing involve linear algebra, since the 
empirical data are often organized in suitable matrices, which describe the proper- 
ties of some signals with respect to others, rich of noise and then less significative. 
Roughly speaking, this idea is behind the construction of the filters, which play 
a fundamental role in most of the mathematical models and of the simulations, 
currently used in the applied sciences. Therefore it is not suprising whether some 
aspects of music and of acoustics are treated in [12l[13l[T4] in terms of linear systems. 
A classical reference remains [TU] for the description of the common techniques of 
the digital signal processing. Among these, the blind signal processing and the 
indipendent component analysis, more properly treated in [Tl[31[3J|31[51[71ISl[n], 
have been largely investigated in the last years for various motivations and many 
algorithms have been developped. 

A significant contribution on the blind signal processing and on the indipendent 
component analysis is [4], where the original ideas of J.-F. Cardoso ff\^^ have been 
generalized, introducing the problems of Exact Joint Diagonalization (EJD), Or- 
thogonal Joint Diagonalization (OJD) and Non-Orthogonal Joint Diagonalization 
(NOJD). lU Equation 3.1] shows that the OJD poblem, originally introduced in 
[3 , may be reduced to a cost function for which conditions of existence and unicity 
(in the sense of [H Theorem 2.4]) should be examined. A similar reduction may 
be done for the NOJD problem, as described in [H §3.2.2 and §3.2.3], looking for 
conditions as in [H Theorem 4.1]. 

Therefore a problem of applied science becomes a problem of optimization, which 
can be approached with the methods of the linear algebra, looking for suitable ma- 
trices which satisfy a prescribed equation. Analogies are very common in literature 
and O [15] illustrate the relations among riemannian geometry, topology, theory of 
compact Lie groups and numerical analysis, when we want to solve some problems 
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of optimization as those which appears in the bhnd signal processing and in the 
indipendent component analysis. In spite of the algorithms which can be written 
with sophisticated computer programs, we will concentrate only on a theoretical 
aspect of an equation of matrices. 

Section 2 recalls some feedback and notations from [8l HI [Til [Eli which will 
allow us to give short proofs of [71 Lemmas 1,2, 3], fundamental in [7| for the EJD 
problem. Section 3 deals with the main result, which is a generalization of ^ 
Proposition 1]. Final considerations and open problems are placed at the end. 



2. Joint diagonalizers revisited 

We begin with some well-known notions on groups of matrices, referring to 
[H [111 [E] for the terminology. 

In the Hilbert space C" of dimension n > 1 over the field C of the complex num- 
bers we have the standard scalar product (• | •) • i^^ u) = ((^i; ■ • • ^n), (jji, ■ ■ ■ Vn)) G 
£n>cn _ xiy^ + . . . + a;„y„ G [0, -|-oo[. The set GL(n, C) of aU non-singular 

n X n matrices with coefficients in C is a non-abelian group, called general linear 
group of dimension n over C. The same is true, when we replace C with the field R 
of the real numbers and consider the corresponding standard scalar product. Con- 
sequently, the following groups are defined GL(n, C) = {B g C"^" | det(i?) / 0} 
and GL(n,R) = {B € R"^" | det(B) ^ 0}, where det(B) denotes the determinant 
of B in the usual sense. 

The set Z(GL(n, C)) = {A,B e GL(n, <C) \ AB = BA] is the center of GL(n, C) 
and it is straightforward to see that Z(GL(n, C)) = {al G GL(n, C) | a G C — {0}}. 
The quotient group PGL(n, C) = GL(n, C)/.^(GL(n, C)) is the projective linear 
group of dimension n over C By analogy, PGL(n,R) = GL(ri, R)/Z(GL(n, R)). 

On another hand, SL(n,C) = {B £ C"^" | det(B) = 1} (resp. SL(n,R) = {B G 
R"^" I det(B) = 1}) is a subgroup of GL(n,C) (resp. GL(n,R)), called special 
linear group of dimension n over C (resp. R). This time Z(SL(n,C)) — {al G 
GL(n,C) I a" = 1, a G C - {0}} and PSL(n,R) = SL(n, C)/Z(SL(n, C)) is called 
projective special linear group of dimension n over C; similarly for PSL(n,R) = 
SL(n,R)/Z(SL(n,R)). 

U(n) = {B G GL(n,C) | (Bx.Bx) = {x,x),x G C"} = {B G GL(n,C) | B-^ = 
B^}, where B^ = B denotes the conjugate transposed of B, is a subgroup of 
GL(n,C), called unitary group of dimension n over C, and the intersection U(n) n 
SL(n, C) = SU(n) is still a subgroup of GL(n,C), called special unitary group of 
dimension n over C. By analogy, 0(n) ~ {B G GL(n,R) | {Bx,Bx) — {x,x),x G 
R"} = {B G GL(7i,R) I B^^ = 5*}, is the orthogonal group of dimension n over R 
and SO(ri) = 0{n) n SL(n,R) is the special orthogonal group of dimension n over 
R. Furthermore, B G GL(n, C) is called hermitian, if B^ = B, and anti-hermitian, 
if B^ = -B. 

GL(n, C), endowed with the topology induced by the operator norm || || : A G 
GL(n,C) ^ \\A\\ = sup{Px|| | < 1} G [0,+oo[ for all x G C", is a topological 
group. The same is true for GL(n, R), mutatis mutandis. In particular, all groups 
0(n), SO(n), U(n), SU(n) are compact groups for n = 1,2, 

All these notions allow us to reformulate more properly the crucial points in 
[H [7] and to simplify some technical arguments. 
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If A G GL(n, C), then we follow [7], defining the off of A as the map 



(2.1) 



off : A e GL(n,C) ^ off (A) 



E 



e [0,+c3o[, 



i.je{l,2,...,n} 



where a^- is the (i,j)-th entry of A. More generally, if G U(n) and 

(2.2) X = (Afi, M2, . . . , A4, . . . , M„) 

is an TO-tuple of matrices in GL(?i, C)™, then we may consider the map 



(2.3) y : [V,M) e U(n) x GL(n, 



k=l 



o^V^MkV) e [0,+oo[ 



and V is called unitary minimizer (or jomt diagonalizer) of [V with respect to 
A^, whenever y{V,M) — Q. In case of connected compact groups, existence and 
uniqueness of non-trivial solutions of the equation y{V,AA) — are ensured by 
classical theorems of calculus on smooth manifolds (for instance, Weiertrass theo- 
rems, see [8]). Unfortunately, in general it is very hard to decide for which {V^M) 
the equation y{V,M.) = is fulfilled (namely, this is the EJD problem in [31 14]). 
Now we consider, instead of TM, the m-tuple of GL(7i, C)™ 



(2.4) 



Xo = {UDiU", UDkU", . . . , UD„,U") 



where U € U(n) and Dk is the diagonal matrix of 
di{k), . . . , dn(k), that is. 



( di(l) 


V 





^2(1) 





( di(fc) 






d2{k) 




/ rfi(2) 






rf2(2) 





with diagonal entries 
• \ 



dn{2) J 



/ di(m) 







dn{k) j 


d2{m) 




dn(m) j 



We have 
(2.5) 



y{U,M^) = J2oSiU"iUDkU")U) = Y,oS{U'^UDkU-^U) = Y,o«iDk) = 0, 

fc=l k=l k=l 

and, of course, the map 3^ is minimized. In this case it is immediate to see that U 
is a joint diagonalizer for y with respect to A^o- 

A more general situation can be encountered when we consider 

(2.6) Mx = (UDiU" + Ai?i, . . . , UDkU" + \Rk, . . . , UD,r^U" + Ai?,„), 

where A e M and i?fc e GL(n, C) (for fc = 1, 2, . . . , to). Of course, for A = (|2J)) 
becomes (|2.4p . Therefore it is very interesting to see (under the point of view of the 
EJC problem) if there are A 7^ for which {U,M\) G U(n) x GL(n,C)'" satisfies 
y{U, M\) — 0. The answer is positive in the sense of [71 Proposition 1]. On another 
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hand, small values of A, close to 0, allow us to study the perturbations in a neigh- 
borhood of the minimum value reached by y on (U,Mo)- Some consequences of 
this condition of equilibrium can be found in [U [31 131 HI HI IS] i where it is emphasized 
the application to the blind beamforming of non-gaussian signals. 

In order to state more properly :7j Proposition 1], we recall that denotes the nx 
1 row vector with 1 in the i-th position and elsewhere, that is, ei — (1, 0, 0, . . . , 0), 
62 = (0,1,0,...,0), e, = (0,..., 0,1,0,..., 0), e„ = (0,0,...,!). e* denotes 
the 1 X n column vector with 1 in the i-th position and elsewhere. This leads to 
the matrix 



(2.7) 



/ ... \ 
1 ... 



V / 

whose entries are 1 in the (z, j)-th position and elsewhere. More generally, 
is the i-th row and a* the i-th column of A G GL(n, C), whose entries are Oy. 

Proposition 2.1 (See [7], Proposition 1). Assume that Ai\ in (|2.6p satisfies the 

following condition 

(2.8) 

yi,j € {1, 2, . . . , n} such that i j 3k E {1, 2, . . . , m} such that di{k) ^ dj{k). 

Then y{U (/ + AG), A4\) — for X ^ small enough, where G is an anti-hermitian 
matrix whose diagonal is null. Furthermore, its off-diagonal entries are 

(2.9) g,, = -J2 /* (fc),<i?fe%- + frj{k)u*R^u,, 

k=l 

where 

(2.10) f.Ak)= J^^'^-'^^'^ ec. 



J:m)-d^il)\ 
1=1 



is essential in order to have a meaningful expression for (|2.10p . 

Remark 2.2. The original version of [3 Proposition 1] is stated with U{I + AG + 
o(A)) J, where J is the product of a permutation matrix with a diagonal matrix 
having only unit modulus entries on its diagonal and o(A) is a polynomial in A of 
degree > 2. In Proposition 12 . II we have specialized J ~ I and have considered only 
small enough A ^ so that o(A) has been omitted. As noted in [71 p. 3 of §1.2], 
if U{I + AG + o(A)) minimizes y, then also U{I + AG + o(A))J minimizes y. In 
particular, this is true in the linear case, that is, when A is small enough and o(A) 
is omitted. All these observations justify our reformulation of Proposition [2TT1 



The rest of the paper is devoted to modify Proposition 12. 1[ involving a wider 
family of solutions, minimizing y. Then we will be able to decompose SL(n, C), 
and consequently, PSL(n, C), SU(n) and U(n), in a canonical way (analogously, for 
the real case). 

3. Main results 

From [m Chapter 3], each invertible matrix with coefHcients in C (resp. R) and 
determinant equal to 1 can be generated by matrices of the form K(l, a) — I + aeij, 
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called transvection matrices, where a G C (resp. a £ R). Note that K{l,a) differs 
from / only in that there is an a in the (j,j)-th position, and, if a = 0, then 
K{1,0) = I. These matrices are important because they generate SL(n,C) (resp. 
SL(n,K)). More precisely, one can see that each B G SL(n,C) (resp. SL(n,R)) 
can be decomposed uniquely as the product of finitely many ^(Ija), for suitable 
a. This is the well-known rational canonical form of a special linear matrix (see 
m PP- 73-76]). 

Lemma 3.1 (3.2.10, see [IT]). SL(n,C) = (/v(l,a) | a e C) and SL(n,K) = 
{K{l,a) \ a eR) for n>2. 

Separately we do the following observation. 

Remark 3.2. off (ii:(l, a)) = \a\^. If a = 0, then off (/^(l, 0)) = off(/) = 0. The 
matrix K{di{k), a) = di{k) + ae^ is obtained by / + acij, replacing the Is on the 
principal diagonal with the di(fc), . . . , dn{k), corresponding to Dk- If di{k) = 1 for 
alH = 1, 2, . . . , n, then K{di{k), a) — K{1, a). Then we can consider the family of 
matrices 

(3.1) Ma,x = {U ■ K{d,{l),a) -U" + XRi,...,U- K{d,{2),a) -U" + XR2,... 

...,U-K{d,im),a)-U" + XRm): 
so that A^o.A = A^A and A^o.o = -Mq. Consequently, we can expect a significant 
generalization of the results in [H |6l [7] , once (|2.6|) is replaced by p.ip . 

Now [Zl Lemma 1] may be reformulated by using algebraic methods and looking 
at y from this new point of view. With the above notations, we consider the linear 
maps 

(3.2) 7,,fe : V e V{n) ^ 7,^-^(1/) = e*V"M^Ve, e C 
and 

(3.3) T,,k ■■ V € U(n) ^ T,,k{V) = e,e*V"M^V - V"M^Ve,e* e GL(n, C). 
Immediately, we deduce 

(3.4) y{v,M) = Y^ l^^^-fcl'- 

Now we consider the linear map 
(3.5) 

m 

S : {V, M) e U(n) x GL(n, C)" ^ SiV, M)^Y1 E ^^J"* ^'^^ ^ GL(n, C), 

where the usual calculus rules (A*)* = A and (AB)* = B*A* in GL{n, C) and the 
equality = V , true by ^ G U(n), imply 

(3.6) %k{V) = {e:v"M^Ve,r - e*V^*(Aff )*(T/^)*e, = e*F*(Mf )*Fe,. 

In particular, if iV,M) G (U(n) n GL(n, M)) x GL(n,R)™, then 7 = V, V" = V*, 
{M^Y =M^ = Mk and then 

(3.7) j:^^{V)^e*V"MkVe,. 
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Once we replace (|3.7p and (\3.3\i in p.Sp . we find exactly Equation 12]. We may 
do similar considerations for the linear map 
(3.8) 



/ 



S" : {V,M) e U(n) x GL(7i,C)" ^ S"{V,M) = 



fc=l i,je{l,...,n} 



A short proof of [71 Lemma 1] is shown below. 
Lemma 3.3. If{V,M) e (U(n) n GL(n, M)) x GL(n,K)™, then S is hermitian. 
Proof. Since ^ is satisfied, (j*,^)" = 7*,;, and T.^kiV)" = T,jk{V) so we con 



elude 



(3.9) S{V, X) = E E T^Jk = E E "l^ok Tl^k = M). 

i¥=3 i¥=3 



□ 



Now we will generalize [71 Lemmas 2, 3], by looking at AAa,\ instead of M.\. 
The condition S{V, Ma,\) = {V, Ma,>.), described by Lemma [3?3l has a physical 
meaning. It is a stationary equation, which allow us to study the perturbations 
around the solution {U,Ma,x) e (U(n) n GL(n, R)) x GL(n,]R)'", minimizing y. A 
more detailed analysis can be done stopping at the terms of the first order in A ^ 
and looking for an anti-hermitian L G GL(n,C) such that, for small enough A ^ 
and a 7^ 0, 

(3.10) 5(C/(/ + XL),Ma^x) = S"{U{I + XL), Ma,x). 
The problem can be more conveniently centered at U, noting that 

(3.11) S{U{I + XL), Ma,x) = S{I + XL,N'a^x), 
where 

(3.12) J^a,x^{K{d^{l),a) + XU"RiU, K{d,{2),a) + XU" R2U, 

,K{d,{k),a) + XU"RkU, K{d^{m),a) + XU"Rrr,U). 

(13.111) is exactly [7, Equation 14], once a = and we stop at the terms of the first 
order in A. 

Corollary 3.4. From p. lip and Lemma [3. 31 for small enough A ^ and a ^ Q, 
(13.101) and S{I + XL,Afa.x) — {I + XL,Afa.x) have the same solutions. 



The following result provides more details. 
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Lemma 3.5. Assume (/ + \L,Ma,\) e (U(n) n GL(7i,]R) x GL(n,M)''". Stopping 
at the terms of the first order m A 7^ 0, the entries of S{I + \L^J\fa,x), for small 
enough A 7^ and a ^ 0, are 

m 

(3.13) Sij{I + \L,Afa,\) = ^ Sijkaijk + {5ijkPijk + <^i]kCii]k)^, 

k=l 

where ctijk, Pijk,5ijk,^ijk ttfe linear maps depending only on K{di{k),a) and L. 
Proof From ^j^ . 

(3.14) T,,kil + AL) = e,e*(/ + XL)" ■ K{d,{k), a)" ■ (I + XL)- 

{I + XL)" ■ K{d,{k), a)" ■ {I + Ai)e,e* 
= e^e*{I + XL") ■ K{d^{k), a)" ■ (I + XL) ~ {I + XL") ■ K{d^{k), a)" ■ (I + XL)e^e* 
= (e,e*/ + e^e*XL") ■ {K{d,{k), a)" ■ I + K{d,{k),a)" ■ XL) 
-{I ■ K{d,{k),a)" + XL" ■ K{d,{k), a)") ■ (/ ■ e,e* + XL ■ e,e*) 
= e,e*K{d,{k),a)" + Xe,e*K{d,{k),a)"L + Xe,e*L"K{d,{k),a)" 
+X'^e^e*L"K{d,{k), a)" L - K{d,{k), a)" e,e* ~ XK{d,{k),a)" Le^e* 
-XL"K{d^{k),a)"e^e* - X^L" K{d^{k),a)" Le,e* 
Since we stop at the terms of the first order in A, we avoid X^eie*L" K{di{k), a)" L 
and X'^L"K{d^{k),a)"Le^e*, then 

T,jk{I + XL) = e,e*K{d,ik),a)" + Xe,e*K{d,{k),a)"L + Xe,e*L"K{d,{k),a)" 

-K{d^{k),a)"e^e* - XK{d^{k),a)"Le^e* - XL"K{d^{k),a)"e^e*. 
Expanding with respect to A, we put 

(3.15) a,,k = e,e*K{d,{k), a)" ~ K{d,{k), a)" e,e* 
and 

(3.16) Ajfc = ie,e*K{d,{k),a)"L-L"Kid,ik),a)"e^e*) 

+{e^e*L"Kid,{k),a)" -K{d,{k),a)"Le,e*). 

so that 

(3.17) Tijk{I + XL) = aijk + PijkX. 
On another hand, p.7p becomes 

(3.18) %^{I + XL)^e*{I + XL)"K{d,{k),a){I + XL)e, 

= e*{I + XL")K{d,{k),a){I + XL)ej 

= {e*K{d^{k),a) + Xe* L" K {d,{k) , a)) ■ (/e^ + XLe,) 

= e*K{d,{k),a)ej+Xe*K{d^{k),a)Lej+Xe*L"K{d,{k),a)ej+X'^e*L"K{d,{k),a)Lej. 

Since we stop at the terms of the first order in A, we avoid X^e*L" K{di{k), a)Lej, 
then 

= e*K{di{k),a)ej + Xe*K{di{k),a)Lej + Xe*L" K{di{k),a)ej. 
Expanding with respect to A, we put 

(3.19) d,,k=e*K{d,{k),a)e, 
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and 

(3.20) e.jfe = e*K{d,{k),a)Lej + e*L" K{d,{k),a)ej 
so that 

(3.21) 7*.^ = + Ae^jfe. 
Collecting all we have found, 

m 

(3.22) s,,(/ + Ai,AA,a)-E E %kT,jk 

k—l i,j {l,...,n} 

m 

= ^ ^ (<5ijfe + eijk\){aijk + fiijkX). 

k=l i,j{l,...,Ti} 

Now ((^ijA: + eijA;A)(Q!ijfe +/3ijA;A) = Sijkaijk + [SijkPijk + ^ijkaijk)^ + [UjkPijk)^^ = 
SijkCtijk + {5i]kfiijk + ^ijkaijk)\ since we avoided the term {eijkf^ijk)>^'^ , as done 
until now. The result follows. □ 

Now we have all that is necessary to state our main result. 

Theorem 3.6. Assume {I + XL,Ma,\) € (U(n) nGL(n,K) x GL(7i,M)™. Then, 
for small enough A ^ and a ^ 0, we get around U G U(n) n GL(n, M) 

m 

(3.23) y{I + XL,Ma.x) = Yl J2 \l^3k{I + XL)\\ 

'==1 ■■■,»} 

where L is an anti-hermitian matrix such that the stationary condition S{I + 
XL^Na.x) — {I+XL,Afa.\) is satisfied with entries Sij{I+XL,Afa.\) as in (|3.13p . 



Proof. The result follows from Corollarv 13.41 and Lemma [3.51 □ 

The reader may note that the cases A = a = and X with a = have 
been already described in |H O [7] and recalled above. The choice of Ma,\, and 
consequently oi Ma,\, is justified by the next result. Preliminarly, we note that 
(3.24) 

^^a,x = {K{d^{l),a), . . . , K{d^{m),a)) + X{U" RiU, . . . ,U" R^U) ^ Ca + XV, 
where Ca e SL{n, C)" and V e GL{n, C)™ in general. 



Corollary 3.7. In the hypotheses of Theorem \3.6\ we may draw the same conclu- 
sions, replacing Na,\ with an arbitrary Af — (Cq^ + Ail?i) • . . . • (Ca,, + XrVr), for 
small enough ai, . . . , and Xi, . . . ,Xr, where r > 1. 



Proof. Lemma 13.31 mention the rational canonic form of a special linear matrix 
C G SL(?T,,C) so we have a unique decomposition of C as product of Cai, ■ ■ . ,Ca^. 
Consequently, we have a unique decomposition of TV as claimed. Theorem 13.61 is 
true for each factor and the linearity of y and S allows us to end. □ 

An intriguing open problem is stated at this point. Some times we can consider 
families of matrices of SL(n, C) which are dense in GL(n, C). When this happens, 
in principle, we may approximate any matrix of GL(n, C) with a suitable sequence 
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of matrices of SL(n,C), for which we have a description by Corollary 13.71 Unfor- 
tunately, we were not able to provide an efficient algorithm of approximation of a 
suitable family of matrices of SL(n,C), dense in GL(n,C). 
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